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Introduction 

This  paper  reports  results  of  the  project  sponsored  under  grant  agreement  number  FA8655-13- 1-3055.  Each 
work  package  is  briefly  disseminated  separately,  to  show  the  overall  progress  of  the  project.  We  attach  project 
related  papers  and  reports  to  give  a  full  and  detailed  picture  of  the  work  undertaken  during  the  project. 
Also,  video  footage  of  conducted  experiments  and  simulation  analysis  is  added  to  complete  the  report.  The 
report  is  organized  as  follows: 
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Additional  material  attached  to  the  report: 

•  papers  published  under  this  grant 

•  papers  written  through  this  project  still  under  review  or  yet  to  be  published 

•  internal  reports  generated  within  the  project 

•  videos  showing  recent  experimental  results 

The  following  abbreviations  are  used:  [WP  =  Work  Package;  T  =  Task;  PM  =  Person  month;  M  =  Month; 
PI  =  Principal  Investigator;  PostDoc  =  Post  Doctoral  Associate;  Grad  =  PhD  Student;  MS  =  Master 
Student] 
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Report 


WP  1  Human  Machine  Interaction 

Herein  we  present  results  in  designing  and  defining  novel  human  machine  interaction  based  on  psychological 
measures  as  well  as  multiple  source  control  inputs,  that  utilize  various  communication  channels. 

T  1.1  HMI  structure  and  design 

[Duration:  Ml -M3;  PM:  2  PostDoc,  1  Grad,  3  MS,  1  PI;  Dependency:  none].  In  this  task  we 
define  and  systematically  present  the  structure  of  the  proposed  HMI  in  the  sense  of  required  hardware  and 
software.  We  investigated  which  types  of  devices  are  suitable  for  representation  of  information  the  human 
operator  is  getting  from  the  agents  in  various  missions  and  through  various  sensory  channels  (force-feedback, 
sounds,  electric  gloves,  etc.).  Results  showed  that  target  location  cannot  be  suppressed  by  focusing  visual 
attention.  Even  when  visual  attention  was  fully  focused  to  a  predefined  target  location,  an  auditory  or  visual 
cue  coming  from  the  opposite  direction  was  still  able  to  capture  visual  attention.  It  is  our  aim  to  make  use 
of  this  automatic  capture  of  attention  phenomena.  Key  variables  that  need  to  be  constantly  available  in 
flight  operations  need  not  interfere  with  a  visual  screen  which  enables  visual  search  needed  to  perform  the 
main  task.  Data  can  be  see  through  during  an  operation,  and  capture  attention  only  in  key  moments  by 
making  a  prompt  movement.  Auditory  information  draw  fastest  reaction  times  (140-160  ms),  followed  by 
tactile  information  (155  ms)  and  visual  information  (180-200  ms)  [30].  Studies  also  found  that  tactile  cues 
can  be  even  faster  than  other  sensory  channels,  when  used  for  alarms  or  direction  cues  (firing  left,  right, 
center)  [31].  Because  we  deal  with  a  lot  of  noise  coming  from  UAVs  during  experiments,  we  focused  on  tactile 
information  in  order  to  translate  information  to  the  operator.  Furthermore,  results  showed  that  speech  input 
was  significantly  better  than  manual  input  in  terms  of  task  completion  time,  task  accuracy,  flight/navigation 
measures,  and  pilot  ratings  in  a  continuous  flight/navigation  control  task  [32].  Utilizing  the  advantages  of 
each  communication  channel,  we  built  the  human  machine  interface  in  the  following  manner: 

Control  inputs: 

•  joystick  control  -  was  utilized  using  PlayStation3  Move  Navigation  Controller,  which  connects  to  the 
PC  via  bluetooth,  thus  allowing  the  operator  to  move  freely. 

•  voice  control  -  was  based  on  CMU  Sphinx  Open  Source  Toolkit  for  Speech  recognition  wrapped  in 
ROS,  using  a  standard  headset  with  a  microphone 

•  operator  motion  tracking  -  was  achieved  with  Openni  tracker,  a  ROS  based  package  for  human  pose 
detection  via  standard  Microsoft  Kinect  RGBD  sensor 

Feedback  outputs: 

•  visual  feedback  -  The  operator  observers  the  mission  status  through  cameras  placed  on  specific  UAVs. 
Visual  display  captures  special  focus  and  attention  when  emergency  occurs 

•  haptic  information  feedback  -  We  designed  a  custom  haptic  rumble  array,  that  can  be  placed  on  specific 
points  of  the  operator  in  order  to  pass  the  information  about  the  UAS  status. 

•  auditory  information  feedback  -  Standard  headset  can  also  be  utilized  to  pass  on  emergency  situation 
awareness  to  the  operator. 

We  were  able  to  design  and  build  the  entire  human-to-machine  portion  of  the  HMI.  We  fused  joystick, 
voice  and  motion  tracking  together  to  achieve  a  successful  aerial  manipulation  task.  We  have  also  used  this 
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Figure  1:  Vibration  motor  array 


portion  of  the  HMI  to  control  a  formation  of  UAVs  in  an  indoor  environment.  The  attached  videos  show 
the  results  of  this  fusion:  [1  -3].  Visual  feedback  was  successfully  verified  in  aerial  manipulation  experiments, 
while  auditory  feedback  was  used  for  emergency  awareness.  We  have  designed  and  developed  the  necessary 
hardware  for  the  vibration  motor  array  shown  in  Fig.  1.  More  details  on  the  BOM  and  board  layout,  as  well 
as  images  of  the  final  product  can  be  found  in  technical  report  [4].  The  necessary  interface  was  developed 
to  connect  it  to  the  PC,  via  BeagleBone  Black. 

Another  aspect  of  the  task  is  to  provide  the  HMI  with  information  on  the  operator  state  (EKG,  skin 
conductance,  HR,  breathing  rate,  etc.),  thus  allowing  the  system  to  autonomously  override  operator  inputs  if 
necessary.  Low  correlation  among  physiological  measures  suggests  that  different  people  in  the  same  stressful 
situation  may  respond  by  changes  in  different  measures,  e.g.  one  athlete  may  display  an  elevated  HR  or  its 
variability,  whereas  another  athlete  may  show  an  increase  in  BP  or  EMG  [5].  Along  with  objective  measures 
of  arousal  (EEG,  GSR,  HR,  BP,  EMG),  subjective  measures,  such  as  perceived  heartbeat,  a  dry  mouth, 
’’butterflies  in  the  stomach”,  cold  and  clammy  hands,  trembling  muscles,  sweating  all  over,  and  tension, 
characteristics  of  movements  (smooth,  easy,  tight)  and  feeling  thirst,  hunger,  cold,  or  pain  (or  painlessness), 
must  also  be  taken  into  account  when  predicting  performance.  After  analyzing  all  the  possibilities,  we 
collected  the  following  objective  measures: 

•  EEG  helmet 

•  ECG,  Hear  Rate  measure 

•  Galvanic  skin  response(GSR) 

•  Breathing  rate 

Besides  afore  mentioned  physiological  measures,  we  incorporated  subjective  measures  such  as  hedonic 
tone,  or  perceived  control  and  accuracy  after  every  task,  along  with  objective  measures  of  success  (perfor¬ 
mance  score)  into  experiments.  We  collected  this  data  of  subjective  measure  of  stress  before  and  after  each 
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Figure  2:  Heterogeneous  symbiotic  robotic  system  deployed  on  a  mission  to  close  a  valve  in  a  disaster  stricken 
industrial  environment 


experiment,  in  order  to  gain  more  insight  into  relations  between  biological  measures  and  true  state  of  stress. 
Through  testing  we  collected  biological  stress  related  data  while  the  operator  was  controlling  multiple  UAVs. 
More  details  can  be  found  in  [5]. 

Portions  of  this  task  are  partly  disseminated  in  [6]. 

T  1.2  Virtual  environment  development 

[Duration:  M4-M10;  PM:  3  PostDoc,  1  Grad,  7  MS:  Dependency:  Task  T  1.1,  Task  T  2.1, 
Task  T  3.1,  Task  T  4- 1\-  In  order  to  safely  test  the  proposed  controllers  and  human-machine  interface, 
we  developed  a  virtual  environment  based  on  Gazebo  ROS  simulator.  Gazebo  simulator  was  chosen  because 
it  is  widely  used  in  ROS  community.  In  this  environment  we  were  able  to  implement  and  test  the  algorithms 
from  WP  2,  WP  3  and  WP  4.  Virtual  environment  was  populated  with  custom  designed  UAV  quadrotor 
and  MMUAS  quadrotor  dynamic  models,  which  were  used  for  controller  testing.  For  dynamic  modelling  we 
adapted  the  existing  Hector  Gazebo  package,  which  implements  quadrotor  dynamics.  The  existing  package 
simulates  wind  disturbances,  propeller  aerodynamics  and  body  dynamics,  and  was  adapted  for  the  specific 
values  of  the  quadrotors  used  in  the  project.  This  way,  we  can  expect  similar  behavior  in  simulation  as  well 
as  experimental  analysis.  We  used  the  VE,  to  create  a  realistic  stressful  scenario  to  train  the  operator  how  to 
use  the  HMI  for  multi-agent  control  as  well  as  to  train  the  HMI  to  estimate  the  stress  levels  of  the  operator. 

T  1.3  HMI  and  virtual  environment  (VE)  integration 

[Duration:  M11-M14;  PM:  2  PostDoc,  1  Grad,  4  MS,  1  PI;  Dependency:  Task  T  1.2].  Given 
the  underlining  ROS  environment  being  used  in  the  development  of  the  project,  it  becomes  straightforward 
to  switch  between  the  virtual  environment  and  the  real  system.  All  incorporated  HMI  elements  are  connected 
with  the  virtual  environment.  In  the  second  part  of  the  task  we  developed  a  scenario  for  training  the  operators 
and  testing  the  proposed  algorithms.  The  scenario  story  was  based  on  an  imagined  event  of  an  unexplained 
breakdown  at  local  Nuclear  Power  Plant  Krsko.  The  operator  was  required  to  land  on  a  valve  and  close  it 
in  order  to  shut  the  power  plant  down  (Fig.  2).  A  combination  of  point  rewarding,  complex  cognitive  tasks 
(Fig.  3)  and  limited  time  frame  gave  us  the  necessary  psychological  evaluation  of  the  operator  stress  level  . 
Results  of  this  task  were  used  in  T  1.4. 
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Figure  3:  A  task  sequence  in  which  the  UGV  carries  the  UAV  up  to  the  pipe,  where  the  UAV  lifts  the  UGV 
across  the  obstacle,  lands,  approaches  the  valve,  takes  off  and  closes  the  valve. 


T  1.4  HMI  and  experimental  multi-UAV  system  integration 

[Duration:  M15-M20;  PM:  2  PostDoc,  1  Grad,  4  MS,  1  PI;  Dependency:  Task  T  1.3,  Task  T 
2.2,  Task  T  4-5\-  Within  this  task  we  wrapped- up  research  done  in  WP1  and  integrated  a  framework  for 
human-in-the-loop  control  of  multi-agent  systems  (MASs),  comprised  of  different  unmanned  aerial  vehicles 
(UAVs),  under  realistic  communication  channels  (Fig.  4).  Augmented  human  machine  interface  that  fuses 
both  classical  wireless  joystick  control  together  with  voice  control  and  motion  tracking  is  put  in  operation. 
The  idea  behind  this  implementation  is  to  allow  the  operator  to  pass  intuitive  motion  commands  to  an 
unmanned  aerial  vehicle  via  joystick  and  at  the  same  time  control  the  aerial  robotic  manipulator  through 
arm  motion,  rather  than  elaborate  joystick  modes.  Added  voice  control  (Fig.  5)  is  ideal  for  single  voice 
commands,  which  are  particularly  important  when  the  robot  is  pushed  towards  the  autonomous  state. 
Furthermore,  in  the  developed  HMI  sensors  are  mounted  on  the  operator  in  order  to  allow  monitoring  of 
the  operator  psychological  status.  By  having  this  information  the  proposed  HMI  is  able  to  decide  on  the 
autonomy  level  of  the  system  independently  of  the  operator.  This  paradigm  is  known  as  the  mixed-initiative 
adjustable  autonomy  (MIAA).  The  HMI  has  been  tested  in  the  experiments  that  involves  autonomous  valve 
turning  operation  [7],  [8]. 

WP  2  Decentralized  Control  of  Multi  Agent  Systems  with  HMI 

T  2.1  Algorithms  for  Distributed  Control  of  Multi  UAV  System 

[Duration:  M4-M9;  PM:  2  PostDoc,  4  MS;  Dependency:  none].  Within  this  task  we  explore  and 
develop  a  set  of  control  algorithms  for  decentralized  formation  control  of  multi-agent  systems.  We  consider 
a  group  of  n  agents  designated  to  perform  a  certain  mission.  Their  interactions  and  communications  are 
represented  through  a  graph  Gn  =  ( vn ,  en)  where  vn  =  [1, . . . ,  n]  is  a  set  of  vertices  representing  physical 
agents  and  en  =  {e-ij}  ,  i,j  G  vn  is  a  set  of  edges  representing  local  interaction  and  communication  links 
between  agents.  First,  a  consensus  based  control  algorithm  is  used  for  decentralized  formation  control  [33] 
represented  by 

±i{t)  =  ^  [ xi{t )  -  Xj(t)\ ,  x(t)  =  -~L(t)x{t) 
ieMi 
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Figure  4:  The  structure  of  augmented  Human-Machine-Interface 
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Figure  5:  The  platform  position  response  on  the  voice  command 
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where  x  =  [x\ . . .  xn]T  is  a  vector  of  agents’  states,  Mi  is  the  set  representing  neighboring  agents  and  L(t)  is 
the  Laplacian  matrix  of  the  graph  Gn.  Consensus  is  reached  when 

Vxi{0),Xj(0),i,j  G  [1 . .  .n]  \\xi(t)  -  Xj(t) ||  ->■  0,  t  -»  inf, 

and  if  the  graph  G„  contains  a  spanning  tree.  In  our  application  this  means  that  agents’  positions  and  their 
respective  velocities  will  synchronize,  i.e. ,  they  will  reach  the  same  value.  Since  we  want  the  formation  to 
track  a  certain  trajectory  in  a  decentralized  way,  we  modify  consensus  based  algorithm  as 

N 

±i(t)  =  ^2  atj  [xi(t)  -  Xj(t )]  -  [xi(t)  -  x*} , 

3  =  1 

where  a,^-  is  the  element  of  the  weighted  adjacency  matrix  and  x*  is  the  position  of  the  each  agent  inside 
of  the  formation  given  in  the  moving  formation  frame.  This  form  of  consensus  control  algorithm  is  known 
as  pinning  control,  and  is  proven  to  converge  asymptotically  if  the  graph  Gn  contains  a  spanning  tree. 
Simulation  results  are  provided  in  simulation  video  [9].  One  of  the  pinning  control  drawbacks  is  that  the 
convergence  is  asymptotic.  Therefore  we  have  implemented  a  binary  consensus  control  algorithm  given  by 

N 

±i(t)  =  y^aijSgn([xi(t)  -  Xj(t)])  -  sgn([ajj(t)  -  a:*]) , 

3= 1 

where  by  using  sgn  we  are  using  only  the  sign  of  the  relative  position  and  velocities  between  the  agents. 
Advantage  of  binary  consensus  based  algorithm  is  its  finite  time  convergence  as  proven  in  [34]  and  small 
local  information  message.  On  the  other  hand,  the  down  side  is  that  it  is  a  bang-bang  type  of  controller 
which  can  be  aggressive  on  the  actuators  and  overly  sensitive  on  small  changes  of  agents  states.  To  overcome 
this,  we  introduced  a  dead  zone  in  the  algorithm  to  neglect  small  changes  of  the  agent  position  and  velocity. 
Simulation  results  are  provided  on  [9]. 

As  a  third  method  for  decentralized  formation  control  we  use  and  extended  a  technique  developed  in  [35] 
based  on  potential  field.  In  this  approach  attractive  and  repulsive  forces  are  created  by  potential  gradient  of 
the  same  form.  The  goal  is  to  move  the  agents  from  an  initial  formation  position  to  a  desired  final  position 
Q  =  Qrit—ioc •  Configuration  space  is  symmetric  due  to  the  agent’s  conditional  identity.  If  any  two  agents 
are  swapped,  the  configuration  space  does  not  change.  This  can  be  described  by  the  action  of  an  idempotent 
discrete  orthogonal  group  T  whose  elements  T(i,j)  are  replacing  the  configurations  of  the  ith  and  jth  agent. 
In  that  case,  the  elements  T(i,j)  represent  a  3N  x  3 N  matrix  that  changes  the  ordering  of  the  ith  and  jth 
agent.  The  matrix  T(i,j)  is  given  as 

1/3  ;  if  a  =  /3,  a  7^  *  A  a  ^  j 

h  ;  if  {a,/3}  =  {i,j}  (1) 

0  ;  else 

Indices  a  and  /3  denote  quadratic  blocks  of  the  third  order.  Moreover,  each  of  these  quadratic  blocks  is  a 
third  order  identity  submatrix.  The  configuration  space  is  not  changed  if  the  third  order  identity  submatrices 
are  located  on  the  main  diagonal  of  the  T(i,j)ap  matrix  than  there  is  no  change  in  the  configuration 
area.  Swapping  of  any  two  rows  of  the  matrix  T{i,j)ap  corresponds  to  the  swapping  of  two  agents  in  the 
configuration  space.  Due  to  a  conditional  identity  of  agents,  the  configuration  space  is  symmetric  and  the 
agents  could  be  distinguished  only  by  their  control  parameters.  Implementation  wise,  the  initial  state  of  the 
configuration  space  is  irrelevant  since  the  control  parameters  are  assigned  after  the  agent’s  initial  positions 
are  defined.  The  control  law  of  a  formation  of  agents  is  given  by  the  equation 

qk  +  bqk  +  ^^  =  0,  b>  0,  Vfce{l,...,JV},  (2) 

oqk 


Page  9  of  19 


Distribution  A:  Approved  for  public  release;  distribution  is  unlimited. 


where  q  are  agent  states,  Uk  is  the  control  input  from  the  potential  function  and  b  >  0  is  the  damping 
coefficient  that  makes  the  system  dissipative.  The  potential  function  of  an  agent  is  defined  as 

Uk(q,qT)  =  ~^uk{dkj,ak)  +  y^juk(rkj,ak),  (3) 

j 

where  the  elementary  potential  function 

uk(p,  °k)  =  uk  (y^j  >  P  €  K3,  crfc  €  M+ 

is  in  the  form  of  a  bell-shaped  function,  i.e.  Gaussian  function.  Vector  p  =  rkj  =  qk  —  qxj  represents  the 
distance  between  the  agent  k  and  target  j,  and  p  =  dkj  =  qk  —  qj  is  the  distance  between  the  agents  k  and  j. 
Parameter  crk  >  0  is  the  control  parameter  which  affects  the  position  and  existence  of  certain  singularities. 
From  the  viewpoint  of  physics,  the  elementary  potential  function  describes  the  interaction  among  the  targets 
and  agents,  as  well  as  the  interaction  among  the  agents  themselves.  Separately  changing  each  agent’s  control 
parameter  effectively  changes  the  interaction  range.  This  approach  has  been  implemented  in  simulation  and 
experimentally  [10]. 

Considering  the  work  done  by  prof.  Lewis  and  his  group  on  trust  based  control  [36],  we  developed 
a  mechanism  for  decentralized  trust-based  self-organizing  cooperative  control.  The  driving  force  of  this 
work  is  the  idea  that  not  all  agents  are  equally  adept  to  take  the  role  of  the  group  leader  at  a  given 
moment.  Depending  on  the  application,  agents  with  the  most  desirable  sensing,  communication  or  processing 
capabilities  should  take  the  leading  role.  Hence,  our  main  objective  is  to  establish  a  group  hierarchy  in  a 
decentralized  fashion  and  reconfigure  the  underlying  communication  topology  accordingly.  The  hierarchy  is 
established  upon  trust  values  towards  each  agent  in  the  group.  These  trust  values  reflect  the  reputation  that 
each  agent  enjoys  within  the  group.  Initially,  we  construct  observed  trust  values  of  each  agent  by  exploiting 
its  local  observations  and  observations  received  from  the  neighbors.  Next,  these  observed  trust  values  are 
passed  on  to  the  mechanism  that  favors  more  accomplished  agents  by  assigning  those  agents  higher  trust 
values.  Since  this  mechanism  is  decentralized,  scenarios  with  unequal  trusts  values  toward  an  agent  need 
to  be  handled  with  caution  in  order  not  to  compromise  the  collective  behavior.  To  that  end,  we  employ 
adaptive  control  concepts  when  negotiating  trust  values.  The  proposed  mechanism  is  illustrated  through  a 
decentralized  formation  control  case  study.  In  extension,  we  present  a  formal  proof  of  convergence  of  the 
proposed  adaptive  mechanism  for  the  network  topologies  that  can  be  represented  with  directed  weighted 
graphs.  The  proposed  mechanism  is  illustrated  through  a  case  study  of  decentralized  formation  control  of 
autonomous  agents.  We  provide  an  in-depth  analysis  of  several  possible  scenarios  regarding  the  proposed 
control  structure.  The  results  of  this  task  were  disseminated  within  following  papers  [11-13].  Simulation 
and  experimental  results  can  be  seen  in  following  videos  [9, 14, 15]. 

T  2.2  Distributed  Control  of  Multi  UAV  System  with  Realistic  Communications  Channels 

[ Duration :  M12-M14;  1  PostDoc,  2  MS;  Dependency:  Task  T  2.1,  Task  T  3.1,  Task  T  3.2, 
Task  T  3.3]  The  main  contributions  of  the  research  undertaken  within  this  task  are  threefold.  First,  our 
analysis  applies  to  MASs  with  general  heterogeneous  continuous-time  linear  agents  (not  merely  to  single- 
or  double-integrator  dynamics)  with  exogenous  disturbances,  directed  communication  topologies  and  output 
feedback.  So  far  researchers  consider  passive  agents  without  external  disturbances  and  balanced  topolo¬ 
gies,  not  dealing  with  transmission  rates,  directed  graphs  nor  output  feedback.  We  do  not  impose  specific 
requirements  on  the  agent  and  controller  dynamics  per  se  nor  on  the  underlying  communication  topology. 
Basically,  when  given  local  controllers  do  not  yield  Lp-stability  (with  respect  to  a  set)  of  the  nominal  system, 
one  can  seek  for  another  topology  or  design  alternative  controllers.  Second,  when  communication  delays  are 
greater  than  the  sampling  period,  no  other  MAS  work  takes  into  account  the  concurrent  adverse  effects 
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Figure  6:  The  final  experimental  scenario  of  the  project. 


of  all  aforementioned  realistic  data  exchange  phenomena.  Third,  the  obtained  transmission  intervals  are 
confirmed  experimentally.  For  plant-controller  settings,  comprehensive  theoretical  and  experimental  studies 
of  degraded  communication  environments  are  reported,  however,  imposed  Zero-Order-Hold  (ZOH)  sampling 
do  not  consider  noisy  data  nor  scheduling  protocols  and  disturbances.  The  results  were  disseminated  in  pa¬ 
per  [16].  Furthermore,  within  this  task  we  experimentally  validated  earlier  developed  multi-agent  formation 
control  based  on  potential  functions  [13].  Video  clips  are  available  at  [10,17]. 

T  2.3  Human-in-the-loop  Control  of  Multi-agent  Aerial  System  under  Intermittent  Commu¬ 
nication  -  Experimental  Validation 

[Duration:  M22-M24;  PM:  2  PostDoc,  1  Grad,  3  MS,  1  PI;  Dependency:  WP  WP  1,  WP 
WP  4,  WP  WP  3,  Task  T  2.2]} 

In  this  task  we  made  the  final,  experimental,  validation  of  a  framework  for  human-in-the  loop  control  of 
multi-agent  systems,  composed  of  several  types  of  UAVs,  with  realistic  communication  channels  that  give 
rise  to  intermittent,  noisy  and  delayed  information.  As  a  part  of  this  framework,  we  proposed  a  novel  design 
of  an  HMI  that  allows  a  human  to  become  a  supervisor,  when  necessary,  instead  of  a  single  unit  operator. 
Furthermore,  the  proposed  framework  allows  the  supervisor  to  interact  with  its  surroundings  by  deploying  a 
dexterous  aerial  robot  as  a  component  of  multi-agent  systems.  Finally,  we  determined  stability  conditions  for 
multi-agent  aerial  system  control  under  decentralized  approach  in  degraded  communication  environments. 
A  scenario  (Fig.  6),  displaying  salient  features  of  this  framework,  brings  together  a  heterogeneous  team 
of  aerial  robots,  each  with  its  particular  capabilities,  and  a  human  operator  in  order  to  close  a  valve  in  a 
disaster  stricken  industrial  environment.  Experimental  results,  presented  in  [7],  verify  functionality  of  the 
proposed  framework.  Video  clip  is  available  at  [18]. 

WP  3  Degraded  Communication  Environments 

This  work  package  is  dedicated  to  robustness  analyses  of  control  systems  (designed  in  other  WPs)  with 
respect  to  realistic  communication  network  artifacts.  Under  the  term  realistic  network  artifacts,  we  refer 
to  communication  channels  that  distort  exchanged  information  and  introduce  propagation  delays,  packet 
dropouts/collisions,  limited  network  throughput  and  communication  range.  The  main  goal  of  this  WP 
is  to  provide  upper  bounds  on  stabilizing  intertransmission  intervals.  As  long  as  these  intertransmission 
upper  bounds  are  met,  control  systems  are  provably  stable  in  same  appropriate  sense.  In  order  to  decrease 
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conservativeness  of  these  upper  bounds,  information  received  in  the  last  transmission  instant  are  utilized.  In 
the  literature,  this  paradigm  is  known  as  self-triggered  exchange  of  information  [37, 38] . 

In  addition,  one  may  want  to  intentionally  impose  intermittent  exchange  of  information  on  control  sys¬ 
tems.  For  example,  when  information  exchange  is  costly  or  undesirable  (e.g.,  in  scenarios  where  somebody 
could  eavesdrop  on  our  communication),  it  may  be  advantageous  to  decrease  the  rate  at  which  the  informa¬ 
tion  is  transmitted/exchanged.  However,  this  decrease  in  the  information  exchange  rate  does  not  come  for 
free.  As  the  amount  of  exchanged  information  is  reduced,  the  performance  of  the  control  system  impairs. 
For  instance,  the  rate  of  convergence  drops  causing  the  control  effort  to  increase  (e.g.,  fuel  consumption 
increases).  In  addition,  the  detrimental  effect  of  propagation  delays  becomes  more  evident.  Therefore, 
intentional  intermittent  feedback  and  its  repercussions  need  to  be  investigated  in  more  detail. 

T  3.1  Enhancing  our  previous  work  with  the  ability  to  account  for  propagation  delays 

[Duration:  M1-M4;  PM:  2  PostDoc,  2  MS;  Dependency:  None]  The  research  carried  out  within  T 
3.1  proposes  a  methodology  for  computing  Maximally  Allowable  Transfer  Intervals  (MATIs)  that  provably 
stabilize  nonlinear  Networked  Control  Systems  (NCSs)  in  the  presence  of  disturbances  and  signal  delays. 
Accordingly,  given  a  desired  level  of  system  performance  (in  terms  of  £p-gains),  quantitative  MATI  vs. 
delay  trade-offs  are  obtained.  By  combining  impulsive  delayed  system  modeling  with  Lyapunov-Razumikhin 
type  of  arguments,  we  are  able  to  consider  even  the  so-called  large  delays.  Namely,  the  computed  MATIs 
can  be  smaller  than  delays  existent  in  NCSs.  In  addition,  our  stability  results  are  provided  for  the  class  of 
Uniformly  Globally  Exponentially  Stable  (UGES)  scheduling  protocols.  The  well-known  Round  Robin  (RR) 
and  Transmit-Once-Discard  (TOD)  protocols  are  examples  of  UGES  protocols.  Apart  from  the  inclusion 
of  large  delays,  another  salient  feature  of  our  methodology  is  the  consideration  of  corrupted  data.  To  that 
end,  we  propose  the  notion  of  £p-stability  with  bias.  We  applyed  the  proposed  methodology  to  the  control 
laws  developed  in  WP  1  and  WP  4  as  well  as  to  the  decentralized  formation  control  mechanisms  and  agent 
models  developed  in  WP  2.  Presumably,  we  focused  mainly  on  linear  control  laws  and  models  because  our 
methodology  is  robust  against  modeling  uncertainties.  In  addition,  the  linear  setting  is  more  suitable  from 
the  viewpoint  of  T  3.2.  Afterwards,  the  obtained  results  were  verified  experimentally  by  using  the  testbed 
and  modeling  developed  in  WP  1,  WP  2  and  WP  4.  The  results  of  this  task  are  disseminated  in  [19]. 

T  3.2  Consideration  of  control  systems  with  zero  eigenvalues 

[Duration:  M5-M7;  PM:  2  PostDoc,  1  MS;  Dependency:  Task  T  3.1]  Within  the  research  con¬ 
ducted  regarding  T  3.2,  we  formulate  and  study  the  concept  of  £p-stability  with  respect  to  a  set.  This 
robustness  concept  generalizes  the  standard  £p-stability  notion  towards  control  systems  designed  to  steer 
the  system  state  into  the  vicinity  of  a  set  rather  than  of  a  point.  We  focus  on  stable  LTI  systems  with  the 
property  that  all  eigenvalues  with  zero  real  part  are  located  in  the  origin.  Employing  the  Real  Jordan  Form, 
we  devise  a  mechanism  for  computing  upper  bounds  associated  with  £p-stability  and  Cp  to  Cp  detectability 
with  respect  to  the  equilibrium  manifold.  Notable  examples  of  this  class  of  LTI  systems  arise  from  consensus 
control.  In  a  self-triggered  realization  of  consensus  control  problems,  each  agent  broadcasts  its  state  only 
when  necessary  in  order  to  achieve  consensus.  Consequently,  the  control  signal  of  each  agent  is  updated 
based  on  currently  available  (but  outdated)  information  received  from  the  neighbors.  Bringing  together 
£p-stability  with  respect  to  the  consensus  manifold  and  the  small-gain  theorem,  we  develop  self-triggering 
for  single-integrator  consensus  with  fixed  and  switching  network  topology.  In  addition,  we  show  that  this 
consensus  problem  is  Input-to-State  Stable  with  respect  to  the  consensus  manifold.  Finally,  our  results  are 
corroborated  by  numerical  simulations.  Notice  that,  from  the  theoretical  viewpoint,  delays  from  T  3.1  are 
incorporated  into  T  3.2,  which  was  straightforward  duty  due  to  generality  of  the  methods  devised  within  T 
3.1  and  T  3.2.  The  results  of  this  task  are  disseminated  in  [20]. 


Page  12  of  19 


Distribution  A:  Approved  for  public  release;  distribution  is  unlimited. 


T  3.3  Investigation  of  model-based  control  and  control  signals  with  saturation 

[Duration:  M8-M9;  PM:  1  PostDoc,  1  MS;  Dependency:  Task  T  3. 2]  Model-based  control/estimation 
and  its  advantages  towards  extending  maximally  allowable  transfer  intervals  are  reported  in  [1],  Basically, 
model-based  control/estimation  outperforms  the  common  Zero-Order-Hold  (ZOH)  estimation  strategy.  Fur¬ 
thermore,  our  framework  is  readily  applicable  to  models  with  saturation.  In  other  words,  the  computed 
transmission  intervals  hold  as  well  when  saturation  is  considered,  but  result  in  local  stability.  In  addition, 
saturation  leads  to  globally  Lipschitz  dynamics,  which  we  find  not  interesting  enough  to  pursue  further  from 
the  theoretical  viewpoint.  Experimental  verifications  of  obtained  transmission  intervals  for  systems  with 
saturation  are  reported  in  [16]  and  [7]. 

T  3.4  Making  a  stronger  connection  between  Cp-  and  Input-to-State  stability 

[Duration:  Ml  6;  PM:  1  PostDoc;  Dependency:  None] 

The  performance  trade-offs  in  multi-agent  systems  are  posed  as  a  dynamic  programming  problem  and 
Approximate  Dynamic  Programming  is  employed  to  obtain  (sub)optimal  transmission  intervals  in  [21].  The 
theoretical  results  from  [21]  are  yet  to  be  experimentally  verified.  A  different  viewpoint  towards  such  per¬ 
formance  trade-offs  is  taken  in  [22].  The  work  disseminated  in  [22]  is  both  theoretical  and  experimental. 

T  3.5  Consideration  of  performance  trade-offs  due  to  intermittent  feedback 

[Duration:  Ml  7-M21;  PM:  2  PostDoc;  2  MS;  Dependency:  None]  As  in  all  previous  tasks, 
our  goal  in  T  3.1  is  to  reduce  requirements  posed  on  resources  of  Multi-Agent  Networks  (MANs)  without 
compromising  objectives  of  the  MANs.  Scenarios,  in  which  agents  of  one  network  are  required  to  maintain 
certain  common  knowledge,  call  for  the  study  of  consensus-seeking  methodologies.  In  order  to  preserve  this 
common  knowledge  in  an  uncertain  and  noisy  setting  and  to  detect  changes  in  the  communication  topology, 
agents  have  to  exchange  information.  Because  each  transmission  and  reception  of  information  necessitates 
energy,  it  is  desirable  to  rationally  manage  available  resources  in  order  to  prolong  the  mission.  Namely, 
communication  should  be  induced  only  when  consensus  can  no  longer  be  guaranteed.  In  the  remaining  time 
intervals,  the  resources  are  released  and  cleared  for  other  activities  such  as  inter-network  collaboration.  To 
that  end,  we  propose  a  parsimonious  consensus- seeking  mechanism  which  draws  upon  ideas  of  self-triggered 
communication.  The  proposed  mechanism  is  inspected  both  theoretically  and  experimentally  for  performance 
vs.  lifetime  trade-offs  using  off-the-shelf  wireless  sensor  platforms.  The  results  of  this  task  are  disseminated 
in  [23]. 

WP  4  Aerial  manipulation 

To  the  best  of  our  knowledge,  to  this  date  multi-UAV  systems  are  limited  to  tactical  surveillance  and 
reconnaissance  missions.  Equipping  several  agents  with  dexterous  manipulators  gives  the  multi-agent  system 
the  additional  ability  to  interact  with  the  environment.  Within  this  work  package,  we  completed  most  of 
the  planned  tasks,  including:  simulation,  design,  stability  analysis  and,  finally,  assembly  and  experimental 
testing  of  the  manipulator  arms. 

T  4.1  Multi  DOF  manipulator  design  and  construction 

[Duration:  M1-M4;  PM:  2  Grad,  2  MS;  Dependency:  None]  Within  this  task,  we  proposed,  de¬ 
signed,  constructed  and  tested  several  multiple  Degree  of  Freedom  (DOF)  manipulators.  After  mathematical 
verification  and  consistency  check,  manipulator  links  were  printed  using  a  3D  printer,  and  manipulator  arms 
were  assembled.  The  material  used  in  the  printing  process  was  Poly-lactic  acid  (PLA),  which  is  a  bio¬ 
degradable  polymer  that  can  be  produced  from  lactic  acid,  obtained  from  fermented  crops.  PLA  requires 
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Figure  7:  Reference  frames  for  manipulator  arms.  See  Fig  8  for  detail  on  overall  UAV-manipulator  system 


lower  temperatures  and  giving  stronger,  more  hard-wearing  products,  which  makes  it  a  good  choice  for  the 
proposed  construction. 

The  proposed  mobile  manipulating  unmanned  aerial  system,  featuring  a  quadrotor  UAV  and  two  4 
DOF  manipulators,  is  shown  in  Fig  8.  Using  the  recursive  Newton-Euler  approach  and  Denavit-Hartenberg 
parametrization  for  forward  kinematics,  each  arm  is  modeled  as  a  serial  chain  RRRR  manipulator  (Fig. 
7)  [39].  The  connection  between  the  quadrotor  body  frame  and  the  first  joint  of  each  arm  is  represented  with 
static  revolute  joint  with  a  constant  angular  offset  for  each  MM-UAV  arm  (Link  B-0).  Denavit-Hartenberg 
(DH)  parameters  of  the  manipulator  arms  shown  in  Fig  7  are  given  in  Table  1,  and  Fig  8  depicts  the  overall 
UAV-manipulator  system.  Parameters  9 ,  d ,  a,  and  a  are  in  standard  DH  convention  and  qj,  qf,  qf ,  and  qf 
are  joint  variables  of  each  manipulator  arm  i  =  [A,  B],  Since  the  whole  aircraft  is  symmetrical,  the  general 
kinematic  structure  is  identical  for  the  right  and  left  arms,  the  coordinate  frames  are  the  same  for  each  arm, 
and  only  the  link  B- 0  is  different  for  the  two  arms.  Reference  frames  are  shown  in  Fig.  7  which  relate  the  fist 
joint  L\  to  the  end  effector  frame  E.  To  make  the  DH  parameters  consistent,  an  additional,  virtual  frame 
Lt  is  set  in  the  origin  of  frame  L4. 

With  Denavit-Hartenberg  parametrization,  joint  frames  are  set  and  direct  kinematic  equations  for  each 
serial  chain  are  derived.  This  procedure  is  repeated  for  both  manipulator  arms.  Individual  links  that  form 
the  arm  are  consistent  in  size,  mass  and  shape  (i.e.  their  mass  mi,  kinematic  parameter  a  and  tensor  of 
inertia  Jl  are  identical). 

Valve,  knob,  and  handle  turning  has  been  widely  studied  for  use  with  industrial  robots,  mobile  manip¬ 
ulators,  and  personal  assistance  robots.  A  typical  requirement  involves  a  grasp  and  turn  of  an  object  that 
remains  fixed  to  the  environment  but  allowed  to  rotate.  Our  framework  and  solution  will  be  evaluated 
by  performing  valve  turning,  which  is  one  of  the  tasks  required  for  the  recent  DARPA  Robotics  Challenge 
(DRC)  [40].  The  task  requires  a  robot  to  locate,  approach,  grasp,  and  turn  an  industrial  valve  with  two 
hands.  Valve  turning  presents  a  challenging  test-case  for  any  system  due  to  the  perception  and  dexterous 
manipulation  required  [41].  Because  previously  mentioned  4  DOF  arms  lack  the  necessary  payload  the  carry 
the  entire  weight  of  the  UAS,  we  redesigned  the  manipulator  system  using  dynamixel  servo  controllers  [6]. 
The  increase  in  mass  was  compensated  with  less  degrees  of  freedom,  yielding  a  2  DOF,  dual  arm  manipulator 
with  somewhat  simpler  DH  Table  1. 


Link 

9 

d 

a 

a 

1 

1a-  f 

0 

3.75 

7T 

2 

2 

0 

3.75 

7 r 

2 

3 

9a 

0 

3.75 

7 r 

2 

4 

Qa  +  f 

0 

0 

7T 

2 

T-E 

0 

0 

3.75 

0 

Table  1:  Denavit-Hartenberg  Parameters  for  Manipulators  [cm].  Showing  Arm  A  only  for  clarity. 
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Figure  8:  Complete  model  of  the  Mobile  Manipulating  Unmanned  Aerial  System 


Table  2:  Denavit-Hartenberg  Parameters  for  Manipulator 
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a 

Number 

(rad.) 

(mm) 

(mm) 

(rad.) 

1 

<7i 
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L\ 

0 

2 

92 

0 

L2 

0 
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With  the  two  proposed  designs  we  were  able  to  evaluate  different  manipulating  tasks  like:  pick-and-place, 
peg-in- hole  or  valve/knob  turning,  which  completes  the  work  planned  within  this  task.  The  results  of  this 
task  were  disseminated  within  the  following  papers:  [6, 24-26] 

T  4.2  Multi  DOF  manipulator  control 

[Duration:  M5-M6;  PM:  1  Grad,  2  MS;  Dependency:  Task  T  4-1,  T  1.1]  Hardware  and  software 
infrastructure  for  the  control  of  manipulator  arms  is  built  to  comply  with  the  proposed  HMI  design.  Entire 
control  infrastructure  is  built  upon  Robot  Operating  System  (ROS),  so  that  the  control  inputs  and  feedback 
from  the  manipulator  can  be  easily  past  between  the  system  and  the  operator  through  the  HMI.  Using 
ROS  allowed  us  to  easily  switch  between  the  simulations  and  the  real  implementation.  Since  there  are 
two  distinct  manipulator  implementation,  each  based  on  different  hardware,  two  low-level  controllers  were 
utilized.  Flyduino  servo  controller  was  used  to  control  the  4  DOF  manipulators,  where  arm  consists  of  a 
servo  powered  rotation  joint,  and  a  DC  powered  linear  actuator  controlling  the  extension  of  the  arms.  This 
is  an  Arduino  based  hardware,  which  we  were  able  to  incorporate  into  ROS  through  serial  RS232  based 
protocol. 

2  DOF  manipulator  arms  are  powered  with  Dynamixel  AX-12  servo  controllers,  that  have  significantly 
bigger  payload  capabilities  but  weigh  a  lot  more  then  standard  servo  motors  used  in  4  DOF  arms.  AX- 
12  servo  motors  come  with  a  low  level  controller  built  in  the  system,  and  use  a  specific  USB2Dynamixel 
controller  to  connect  with  PC.  Transport  layer  based  on  USB  protocol  is  wrapped  as  a  ROS  interface,  and 
makes  it  possible  to  connect  both  manipulator  setups  to  the  same  HMI. 

Finally,  the  interface  between  the  operator  and  the  manipulators  is  achieved  through  motion  sensing 
using  standard  Kinect  sensor.  Through  interpretation  of  human  poses  and  motion,  we  are  able  to  map  the 
human  moves  into  manipulator  pose  control  inputs.  This  allowed  a  natural  and  intuitive  control  of  the  arms, 
while  performing  various  manipulation  tasks. 

T  4.3  Mobile  Manipulating  Aerial  Vehicle  mathematical  model 

[Duration:  M7-M9;  PM:  2  Grad;  Dependency:  Task  T  4-2]  The  end  goal  of  the  mathematical  model 
is  to  combine  both  manipulator  dynamics  and  the  dynamics  of  the  quadrotor.  Due  to  the  resulting  complexity 
of  the  model  and  keeping  in  mind  specific  mission  requirements,  the  quadrotor  dynamics  considered  in  this 
paper  do  not  account  for  various  aerodynamic  effects  (i.e.  blade  flapping,  ground  effect,  etc.)  experienced 
during  highly  dynamic  flying  maneuvers.  Most  of  the  missions  that  include  some  form  of  interaction  with 
the  environment  require  stable  hovering  maneuvers,  which  justify  a  simplified  mathematical  model  without 
accounting  for  the  previously  mentioned  aerodynamic  effects. 

As  the  manipulator  dynamics  are  introduced  through  the  recursive  Newton-Euler  method,  it  is  possible 
to  separately  model  the  quadrotor  motion  based  on  Newton-Euler  equations  for  rigid  body  translation 
and  rotation  [42].  The  two  models  are  then  combined  to  yield  a  complete  model  of  the  proposed  mobile 
manipulating  unmanned  aerial  system.  The  resulting  equation  is  highly  nonlinear.  Previous  work  in  this  field 
has  concentrated  only  on  load  mass  stability  problem,  ignoring  the  coupled  manipulator  dynamics  [43] ,  [44] . 
For  a  1  DOF  gripper  tools,  this  assumption  is  welcome,  but  for  4-DOF  arms  that  introduce  a  significant 
increase  in  payload  and  moment  of  inertia  this  simplification  cannot  be  applied.  Two  underlying  effects 
are  identified  to  directly  influence  quadrotor  control  are  the  change  in  the  overall  center  of  mass  CM  and 
moment  of  inertia  J.  Building  upon  the  results  from  [45,46]  we  present  a  complete  mathematical  model  to 
establish  a  stability  criteria  for  this  mobile  manipulating  unmanned  aerial  system  [24]. 
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T  4.4  Mobile  Manipulating  Aerial  Vehicle  control 

[Duration:  M14-M18;  PM:  3  Grad,  2  MS,  1  PI;  Dependency:  Task  T  4-3]  This  task  is  tightly 
coupled  with  the  development  of  HMI.  The  teleoperation  part  of  the  controller  was  successfully  implemented 
and  tested  in  three  benchmark  manipulation  tasks.  The  controller  is  devised  based  on  the  nonlinear  dynamic 
model  of  the  system,  where  we  identified  four  parameters  in  the  attitude  control  loop  of  UAV  that  change 
during  flight  and  manipulation: 

•  Km  -  Propulsion  system  gain  changes  drastically  through  time,  especially  in  load  manipulation  missions. 

•  Tm  -  Propulsion  system  dynamics  changes  due  to  various  effects  as  the  mission  progresses  through 
time. 

•  [3  -  The  aerodynamic  conditions  constantly  change  during  the  flight. 

•  J  -  Moment  of  inertia  changes  depending  on  the  load  and  manipulator  pose. 

Aerial  manipulation  missions  mostly  require  steady  flight  conditions,  for  which  the  changes  in  the  aerody¬ 
namic  conditions  as  well  as  the  aerodynamic  coefficient  /?  can  be  neglected.  On  the  other  hand,  if  the  battery 
power  supply  is  kept  constant  throughout  the  mission,  the  variations  in  the  Tm  can  be  minimized.  The  two 
remaining  parameters,  J  and  Km  diverge  the  most  during  aerial  manipulation.  The  variations  in  the  moment 
of  inertia  have  been  previously  discussed.  The  propulsion  system  gain  changes  are  mostly  caused  from  the 
variations  in  the  load  mass,  which  changes  the  piecewise  linearization  of  the  quadratic  relationship  between 
the  propeller  thrust  and  the  applied  voltage.  Apart  from  that,  the  variations  in  temperature  and  the  battery 
depletion  also  change  the  linearized  motor  gain  throughout  the  mission.  In  order  to  adapt  to  parameter 
changes,  we  implemented  and  tested  adaptive  control  both  in  simulation  and  real  system.  Opposite  from 
our  expectations,  once  properly  tuned,  the  system  proved  to  be  stable  even  without  the  adaptive  control. 
In  situations  where  the  parameter  changes  cannot  be  predicted,  we  proposed  and  tested  a  model  reference 
adaptive  controller  (MRAC).  Using  the  Lyapunov  stability  theory,  the  controller  is  proven  to  be  stable.  The 
simulation  results  proved  that  the  MRAC  is  capable  of  stabilizing  the  oscillations  produced  from  the  unstable 
PID  attitude  control  loop. 

To  better  utilize  the  MRAC  and  Gain  Scheduling  adaptive  capabilities,  a  hybrid  system  based  on  a 
switching  automaton  was  proposed.  The  hybrid  system  defines  four  phases  of  aerial  manipulation  missions: 
Flight  phase,  Arm  Deployment  phase,  Manipulation  phase  and  Adaptation  phase.  The  system  starts  in  the 
flight  phase,  where  the  MM-UAV  flies  to  the  designated  point.  During  arm  deployment,  gain  scheduling 
algorithm  is  crucial  for  keeping  the  system  stable.  After  the  arms  are  deployed,  the  UAV  switches  to  the 
adaptation  phase  and  starts  the  self  induced  oscillations  in  order  to  fine  tune  the  controller  using  model 
reference  adaptation.  During  the  manipulation  phase,  the  shift  in  the  center  of  mass  or  the  change  of 
moment  of  inertia  produced  from  interaction  with  the  environment  or  from  picking  up  an  object,  could  drive 
the  vehicle  unstable.  Therefore,  the  vehicle  is  allowed  to  switch  back  to  Adaptation  phase  if  such  a  problem 
occurs.  After  successfully  picking  up  an  object  and  before  flying  back  to  the  base,  the  air  robot  once  again 
changes  to  the  Adaptation  phase,  before  flying  off.  Once  tuning  is  complete,  the  aircraft  switches  back  to 
flight  phase  and  flies  back  to  the  base.  More  details  about  the  gain  scheduling  and  model  reference  controller, 
as  well  as  the  automaton  can  be  found  in  [24,25,27]. 

T  4.5  Mobile  Manipulating  Aerial  Vehicle  experimental  verification 

[Duration:  M19-M21;  PM:  1  Grad,  2  MS,  1  PI;  Dependency:  Task  T  4-4] 

The  proposed  controllers  and  HMI  have  been  tested  throughout  experiments  with  the  unmanned  aerial 
system.  The  experiments  conducted  within  this  task  proved  the  validity  of  the  proposed  aerial  robots.  We 
have  selected  three  benchmark  manipulation  task,  to  verify  the  feasibility  of  aerial  robot  concept.  The 
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three  chosen  tasks  are:  Peg-in-hole,  pick-and-place,  and  valve  turning.  These  tasks  exhibit  various  aspects 
of  aerial  manipulation  problems.  For  instance,  during  peg-hole-task  precise  navigation  is  required  in  order 
to  complete  the  mission  safely.  Unlike  valve  turning,  during  peg-in-hole,  there  is  no  full  contact  with  the 
environment.  Therefore,  we  use  valve  turning  scenario  to  verify  problems  arising  from  full  system  contact 
with  the  environment.  During  pick-and-place  there  is  an  obvious  discrete  change  in  the  dynamics  of  the 
aircraft  (i.e.  abrupt  change  in  mass  and  moment  of  inertia  due  to  payload  increase)  that  the  controller  needs 
to  compensate.  Proving  that  the  controller  works  for  all  three  cases,  we  have  verified  the  proposed  concept. 
Another  experimental  verification  within  this  task  was  to  analyze  the  proposed  HMI.  We  have  conducted  a 
series  of  experiments  that  showed  that  the  multi-channel  human  machine  interface  alleviates  the  operator 
burden  and  allows  him  to  complete  the  mission  safely  and  efficiently,  which  is  in  part  disseminated  in  [6]. 
Furthermore,  we  are  attaching  the  following  videos  to  show  the  experimental  results:  [28, 29] 
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